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(57) ami 

A#*^St%^te iiST"Mo«Jt <k < AH© 

u mmmmtmmmm^ zmmwt&fc+z cue 



I (A-1 ) 



I (A-Z) 
I (A -3 ) 




H < , <<• )lGll] 

[t»*is i ] wmfrb>\(Dm%mftT%tz ^nmrn^ 
atev^t, A©n©ff« fflsf*tt«»igi«£»5£ 

iio 

[w*n 3 ] s«^6Ao«*am , r«fc ftoam^ 

&t*5fT, W&<D&<OBtf.*LtzT>7U-htW& 10 

t ® v ^ f - y ?m % ft * l s v -v ^ y ?m © m t s f r 
> 7 b- h * l , m t a ^ o v «, > *^46 

5£A6fc L # f «W±T7*fttf , HWS ft ft t>71/-F 

[is*un] mmmmmmx, 

£© l #7C© kX h^A%*46, UtxF 9=7L.<m 

m"ZMm®*m%?% z t wmttzmmm i &f 
l 3 ©v^-r n* 4 1 «fcE«©H»m£&> 

[»*gf5] ffineff&ttffittn-F, 6**-«v<7*- 20 
£ © 2 -jRjtto tXh^A^ffe, IliXh ^7AOi 

«^6«««*ft* , r« c t sw*!* i &f 
l 3 ©f im ^ i iSicie«o«ai±5^& 

[fi*H6] i£Mit^7^-^Cl^©iiX 

©f f ftfr 1 3StcSSK©liatti^ffic 
[11*118] MiefecO^*tLTfeffltBj!gcD2^ 30 

[ll*Jl 9 ] ffliE8«ffif£«©£ligE&S£«Ffr£>« 
^Jgfc&^-f Set t -T « ll^il 1 * I ^ L 3 Of 

[is*n i o ] MmwMwmmz>mmc #»ju * 

ft?ftO>J>1K«iT©!|$S»%:R«>, 

*« 1 4tu3©i-fw i mamnim&ii&o 

[Mflll] tyE'J^ilt7tfiJ-r?.c:ttJ;'3, 40 

ii{j*^»$tisc t a swssi o 
titers 1 2 ] t LX'hmmo 

1 3 ] HufEfe©»ffi«t LTfeffl i:9§£© 2 

1S*H1 2{Cieic©H«3ffi7:ffio 

[»*« 1 4 ] mummm t vr^mmic mf^m 
&®mmmm «tc fcttaw & t jtr a &o m©« t ^ so 



1.VI!(]t : 8 - 6 3 5 9 7 

2 

fitw^c t^tr^t -r 5ii*ri i o tfctt i 1 1 
saso&ttaj^iio 

tefi i t 'jL xt^ L. l _ iLT> & 

m^zctzimt-tzn *« 1 4 tie« ©nwitBTi 

;Sc 

[if*m 6] i»E#»«fcLT/h{S«o£HJii£a 
ttm.K*m vsc tttrmttmmm 1 0 $ 1 1 

[is*m 7] Mte'j^«©sia^ffiEa«ttstLT 

feffl©SIBJBift»JS»«rffl 1^4 C t fct5W*3S 

Wnm \ 8] 1 mm- mini 'c 4 HHKli! 

««*j*^-r 5 C t ^r#a t -T SttSffi 1 *f L 3 Of 

i: %t#a i: "T 5 W*iS 1 4 f L 3 ©l >-fft ^ l iliclBK 

[M«3f 2 o ] mmm § ©wn^ata 

t ^r#a "T 5W*3l 1 4 f L 3 Of "f ft *M tm 

[n*3i2 2] means mummmz 
zctwmttm&m 1 4i^3«^fwist 

[|»*H2 3] lnjieiIfg^fiMi*^ft, P.HMffiii^^ 

Sdt^gri; 1-511*11 1 %f L3©f WmiC 

IB«©Mttffi^ffic 

[is*H2 4] -mmmmmtmx, wf-Y-y^A, 

II 1 fcf L 3 ©f "fftfr i iitfe«©M«jm^?Sc 

[iii*ri2 5] mmmmim?, nmmitxtb. m 

fcr&ct *w$nttz>mxm 1 *f l 3 ©v^-rti^ 1 
JS(cieig©ntti±s^£ 0 

Sit^ffll^Ct L fcll*H 2 5 t|E«©M 
[11*31 2 7 ] 1I*H 2 5 £ fcl± 2 6 Iz ES©|«tt)l±l 



UhiVii 2 8 ] $mm l % O L 2 6 W>f ftfr 1 ISK 

[0 0 01] 
[0 0 0 2] 

fiWcA*ffl»Lfc*5-?J*©it£fc{i, AOfiOfetf 
3HEfc%5 J; 3 tc^lts t -Ifttc^K^afcAtc^x 
§ Jg tftffi < , 2§:H©H£ « 6«ci:K5c 
[0 0 0 3] ^JtaUgOJft^, tlU^ff^i^-^Tfe 

ifcfci^\ *Pf;(c(iie7 i cTiSii5$nfcJiiiHi^xhD# 20 

£m^T«i5£*lfdBiiBHStgtf 1 #©7^;l/A©^lciI 

fcfe t «nia it c t {c^emw mom^tm^ s-t 3 ©# 

Sf*L<, ccO^Mitn l ' s 1 
ofc^XCiiI^t(iA©W©fet*g-r5©AMIfiJTfe 

tttttt b fttzfeV^Mlc tolf 3 A©«©fetfflafefc 
45 J: 5 fc»5t**i*«>* C 3frb TfcSo 

[0 0 0 4] c©«J:5fcA©«©fe*S$i: LT&tttt 
"•• :V: V'i::^t5^;£:jf^5tte 1 nTtSf), £ 30 

5-7 -c comm. mm ©-aw** i »«:■*-«» 

[0 0 0 5] fctxtt; *5-7 Y^AO^Wi^cOlS 
x- * ^tttB L, c ©tttti L fc»*r-*fc S^TH 

[0 0 0 6] f/t, KH«*©A©8i*Miffl-r5;58;i: 

(ItRPg 52-156624 ^&$g, #BPg 5 3-14 
5 6 2 1 ^$E, ^PjSBS 53-145622 WAffi » 

£&»fc,6*R G50 , G (») N B (if) ©3fefc# 

[0 0 0 7] t bt, #P ff W 4 - 3 4 6 3 3 3 -^#8SC 
ti, ffi^L/cx-^fefflffi (H) , g£« (S) icm 50 



S&U H, S©2^7xbX h^ASrfNtU C©tX 
[0 0 0 8 ] 

imwm&L& 5 ft* mm iMot^u-^^m 
■«f©siww*iB«^4 < c t an?* 

BHS«{C * ^ lx- * #5 >f h ^ >T'PJB« l & «■ 
ntfft6!8:i^ii>» ^Wt^*(cB#p B W^^5i:^5P«g 
gtf&5 0 Sfc, C©7Jj£©±I£, t^U-^^SL 

4i^£4fcAfk (i»{t) ti^pJArr**. 
[0 0 0 9 ] HWi^bBflfer-^^atBL, m 
fefBH i: ¥ ij Br S ti fz m9t& O^^x^^mtt^moD 

[0 0 1 0] ?lt, Hlf4-3 4 6 3 3 3^^$StC 

wzmm* set an? * * v sti^ hb*^ 

5o 

[0 0 1 1 ] *5!W(i±8EOjiStA^3b , *Tft$nfcfc 

b, A^^s-r^ti«n?ao«s«fc < Aoas*«i 

[001 2 ] 

IBMZffl i*"T 5 fcJ6cD¥ IS] *mWte± ffi© g eu^ria 

at fc^T , A©»©ffj« t ti a t «n« ffi^^^s 

[0 0 13] BiJO^ffiKAtltf, H«^6AO» 

©ii$T>*ffl c 1 1 j: '2 mmmmmz ^a-r § «k 5 
[0014] $bt, Bijo^ffitAnar, h»*»6A© 

p.^rWimf 5 fcfcCttffiTj SicfeOT, ffi ftfflfcK 

h t mm t <D-? y * y {TSSttlL, v 7 f - > 
>y&ft : flsbfetbrz L t WifiAJ;T$>tl«\ 3B«?Stlfc 

x>7u- h mmm*m<mmmttz, a a (at 

[0 0 15] 

«fr bHM^*i*^-r s cttiD, * 7 -BMIi 1 b 
A©P?:ffltB-r5c 



[0016] %tz, mvttiic&titf, wmfrt>x<Dm 

[0017] jbt, gijo^ffifc^ntf, m'KDmmi 
vtmmtw v mm u -y ?- > ^aoa 

t fll^r >7° U- h L , IfeSA^f:? -y ?■ >^ 
fi*^ feSftfc LtHiW±T&ft(|\ SiRSftfcf-> 

7° b- h © fiti^/iiiiiiti $ 1 * 5 c t tc i »? , * ^ 

[0 0 1 8] 

[0019] na, *mmic&%mmmm*m^ 
mmmtm i©7n -y 2 0 t$> s 0 

[0 0 2 0] 7^;kA, 1 tt*7-HWfctflE«Sftfc:7 
^;I/AT$t) s ^ f 7 A T$ o T i * v 7 ;b AT 

£ o t t ^ $ to * i \ # «^ b safe*!* tb-r 5 1 t 
{i#s/*oSij}tffi^6iI«flllfe*«ffi-rti(fi <, 
fi^ 6 me* jaavf s t £ £ * ^'©ii^f* * vt 

L TflHfe«ra tti L T & «k l> L , * 5? ©«Jt« bilEJMl 
feJ&WtHLT&iW X+ft2li7-f;l/AlOA7- 

(if) > G (10 , R (#) c 
© B G Rffltt , 4H68S 3 -pflMBSftfcftte A / D £818 
4T*-r->**;bx-#fcg«£*lTC PU 5 1 7\fi £ ft 

s» cpus t«, fta-*-s«ttHioftfto*ffla** 

[0 0 2 1 ] H2ti, *aWt«k*»«ffi^jSO*l© 

[0 0 2 2] £?\ 2t<toT#btlfcS-li^ 
©BGRflfrb, <9i«, feffl, felt, (B- 

R) ( i K i „ -l J u , "L J 'A- * 

// s ((f (x+u,y+v)-f ') • (t(x,y)-f))dxdy 



1) , Cnb©fe©#ft:1^46^i6fclEHrt(CAoT 
l ^tlfcf , it MBRtfflfeT & § t ¥ iJSf § CA — 2) o 

fc, sK^HWOB G Rffl^ 5mfe*«fflT 

[0 0 2 3 ] f]nfet1 ; ^Stl/;ES^^'3l««ji](f 
titLTX7 ttJSrfr* 5 (A — 3 ) „ x -y v^ffi7? 

©M^mSfllJ: 0 fc*:£t*ftlf , Jt*if 3t£x-y "Jwm 

[0 0 2 4 ] Xf7 7" (A -4) £t\ 

5M/ffiM (i a/i b) ojt*©s43i*aotsna& 

ZWtPiB<D&T>7l>- h (13#I) fcfMtfSo 
C ft b ©P. r >7° U- h tt * iMf^t L T fc t IB1I»B 6 

C^L!i;Fq©$S?|5^S^T-2 flfk^ftTt^o M% <DM<D 

U-h©|i?(5{3;, i$;Si^, »SL<tt2~3H*<0(B«r 
fcfc^T, ^P^©P,©$iSH fc ©V -y y «%±lf T fe 

[0 0 2 5 ] Xf7 7" (A-4) Tfi, nt^T, lr>' 
■A i»x >7°U- h © v -y i7"Jt** ft *. v y f- 

lfS2n5J:v4faT'77?^lm (u, v) % 

[0 0 2 6 ] 
[Si] 



m(u,v)= 



v / // s (f(x+u > y+v)-f') 2 dxdy// s (t(x 1 y)-t') 2 dxdy 



h^^L, sat (x, y) oimrnzm?* f , t* 

(i^ft^'ftf (x + u, y+v), t (x, y) OSrt 
[0 0 2 7] C©ckd%#aT\ MlliClSryyb- 

h^ffli^Tv-y^y^tf*^, «td:<-a-r57 : -> 

7 c b-h*74t<Biilt^LT5)?to (A-4) , ^y=f-y 
fR«*SiS«««iT**Si:¥iJ^rs (A- 6) o 

[0028] tit. mmm^^m^ treble, mm^± 

mmt-o, ^H^^ttc-fbLT^oT, 
v-y ^>^jt%*46, -?y?-yymtfmi£mu±cDitim 



[0 0 2 9 ] H 2 JCS Lft » 1 ©^ffii?ijt cfcfttf , «© 

ttttSf « C t *<T * § 1 1 >5 5 o 

[0 0 3 0] ffll&*«Ottmtfefc5mfei:#4 

[0031] *bt, m&©seh*, *#wiffl, 

i^a« ^ b T fe Sf Ott ffl JSC Blft 1 4 3 i: -5 ®$k fe 

[0 0 3 2 ] 4*5, 0 2 (C ^ Lt-m 1 0*aE»JT*(iilfe 
MS©«)tB o T^f-Ir 771/- h t © V -y ^ 



J; IV 

[0 0 3 3] $fc, II 2 iCTS LfcH 1 0*«ff9T«xy 
zs-V3b S g T 2 iff fc L Mi i x -y v© glir > 7V 
- h ©V -y ^y^ff-a fc AV *fmii H tltll b 

fit, flRfe^S^T2fiIftL/cllT->7 e b- hi:OV7 
f-^'^ToT&Mv 

[0 0 3 4] tc5T\ ^1 ©Hlij{?iJ!c «k o TPJtM 10 

*s t mizi£ft tzw^m t * * lt t +#t $ * t>\ m 

[0 0 3 5] H4tt, *«WtJ:««ttai^ffioa20 

[0 0 3 6] $-f, aiO*««Uc±-DTWS*tlfc« 

?%> (B-l ) o COtt, !H8tflST*2&5i8£ (05 
(a) #P,BJ (13 5 (b) #&) ©^ft 

a\ Httoiqi * mm t t 

g fill Wet f3£L, -*f, illSOftlttfttQil 

f -rats, «««nw«te*#UT, ±¥#, 

5h £¥#*B«JiS* c©g<s 

*I«W©K5££iSti, t -a fc & < LT t> fr£to& 

i\, tztz-is. wmoftztmo®^ meic^t&v 30 

fci:t, SiStfO. 5-0. 8©SEH*S^llpl^tS5 
[0 0 3 7] ^fC, ^n?'n©@^|{SMl*lT% I7t 

x 5 km&wfmxhy? (b- 

2) o CCTtt, HS*8lf-yh-eail/fc»£fcO^T 

wnats (w*o : 0, mm 1 0 0 : 2 5 5 ) . 

&MW (0- 2 5 5) *K|«{c8^fiJLT, ^n^fn© 

mmmmpiT © ajgyss** 11 <om ft&mmfi 

fgl^©T\ txF ^7itt'>4< t feJRBOWSO»» 40 

tiife©i^it©gi5^(ci;-^^-r§ (H7#bs) o c 

[0 0 3 8] ST, CoLTft&LtzKZ.h'y^L.OB 

•f, )iafi©fltf fcStoti^BJf^igH GfcfcAlf 8(£<y h 
796 W±) Ttl*© tf - ^ (SIO t - >7 ) *gfr a 
^tcm«««k 0 fefiv W«HT'ft*:Otf- ^ (H 2 © 



ofr<Dl©tf>7;V*aBfc L*lS»:*i6 5>. (H2© 

/ (H 1 ©tf-^O-JM) ©fl^ftm 
U (B-3) „ 

[0039] cmmmtfmzmmm'Sbnn (b- 

4) , c©LXh^A©ff5««fI^LTi^fc«lfr 
•fSo Xf7^ (B-4) Cfclt^pfr^lEHii:, Sfill 

£ SJilG H ^5l<i6 T fefr * it n tf % b % l V 

[0040] c©ck oicLTs znz'fttDg mmmm c 

BHR*«*tt«T»**i:ipJ£L (B-4) , SfcWB'T* 

[0 0 4 1 ] Sfc, H4t^LfcS2 0*««»jTtt, X 
- 1-1 ill, _ IB 

- 2) T^©gM^M(*l©H^It©bX h V^yLKim 

t\ 

[0 0 4 2 ] H 4 fcaRLfcSS 2 ©**«IK ifttf, IHit 
[0 0 4 3 ] t/c, fl^xM*, £©?}££<£ Dg«J 

«t lift© [S] t icttff«ii^9»t.* 

So 

[0 0 4 4 ] ftfe, H4fcSbfcS2<0*««ilTH:W* 
© 1 *7C<0 t X h ^5A*flF*Lfc*', *»W{4cntC 

iBe-r, ittxn. mmt&mt*mcLrz2&Tttx 

©^SSfiJ t m m -r S J: •? t LT t> J; v L, Hj^ 

StUg© 2^7tbXh^7A J f», feffiti5S©2^7c 

[0 0 4 5 ] 118(4, *^H|jiCj:S»m^?£©ii3© 
*i5S M © 7 a - ^ + - h T h 5 c 

[0 0 4 6 ] S-f, ^1 0*»WfcJ:oTW£aftfc« 
«*MW*rtOB«*W*fc:S»-r« (C - 1) „ *D?fiS 
«lT-tt^ftffiSi«rt©iBf6*WSfi:SSS-r 5 tOi: LT 

1MB «{c Wfc L T t S to 4 IV 
[0 0 4 7 ] P.^ffiSS«©^f ^Pt^Xl: 
^■to^Sfcfetcffi^ !?I/jN^tTi\ P,I^SfMi»S©-9-YX 

*«*{fr* (c-2) . ^lt, «tt{t«nfca»8ii 

ffi«{cMLT2^ft7-UxS^fT-5 (C-3). 

[0 0 4 8 ] CCT\ C©7-'Jx|g^C)SlltC*5l^ 
T, WW-X^^ h^A^Sfe^f ^If-^ffiTM 
ttitUS<. ?LTv iE!;»lt^(o*o/^ 

2^5t7- U x^it^ff o tO*^ 468Stt^H 6 irSd 



^^itLifti) (C-4) = 

[0 0 4 9] LT, -©#!*«# LflMIW hT?**l 
{* (C-5) 



(C- 



6) . 



[0 0 5 0] tC5T\ H8tSbftS30**ffi!Tli 
Xf -y 7 ( C - 3 ) T* 2 ^tc 7- U iSM fi-otzt>\ 

M\ fc£L, £©££■£«, MOW^^f— & t LT> 10 

[0 0 5 1 ] H 8 fc * LfcS 3 Qgflfcffllk: «k hfcf , ffi£ 
ffifIlt®ttai©^IE5t$tJ: 0gM©g[Hfi (gJf>p) (DftL 
mtfi-ftiX LSo/ci'? Slptb, * 

[0 0 5 2] %to\ [18 LfcSI 3 ©*Sf§MT(i 2 # 
7t7-U xg&fcffofctf, *fg^fi; 2 #te7-'J 
*&fT*>4<Tfc, JBifi»«Ptttlp-f*tOT*ntf3&» 20 

[0 0 5 3] HfltOttN. ««*Wfftxi, 

i; y tt^OiSIBO^ft^ftte Ot^T* © tf- f ffl 
?# 6 ftfcSJP x- * fc ©HT' v 7 f> df*«j 5 , L # ^ 

<n>&<, feffl, feit, i^nofe^n-r^fua^x 

[0 0 5 4] 2*7C7-'Jx««K:Jt'«TW 

[0 0 5 5] H 9 «, *58W K * SUHftHW i£©lg 4 © 
Sfe»J©7 a h T** §o 

[0056] s?\ ^io*ffi«ijjc«fcoTipj^snf=as 

«£Miltl*J© &«©&!!, (D-l) , fe 

ffi, B^jt©2^7t^r a i ]0±t / ^SSIIP3©il«^*i6, ctl 40 
fc^iftMfcf 3 (D-2) c 

[0 0 5 7] 7\7-";7° (D- 2) tfttflffl© 
llllgi;/ Mfi^lGHrtTSntf (D-3) , ^©/M^ff.^ 
iilTSS £ Iij^-T § ( D - 4 ) „ 

[0 0 5 8] £fc, MHBtfltRCftfrOfc, fe^H (L 
* a' b" , L" u' v- SS) T-©#»«P3{*©tt:!*, 

[0 0 5 9] H 9 toT L fc£ 4 ©HMUiC «k tllf , jiff 50 



[0 0 6 0] Sfc. 04tSLfcS2 ©*ffi0IJT*#i& b 

tif=i*sui*ffl*^toH*Tfiji8frnif» o isii ,ti 

[0 0 6 1 ] HI 0»*5883fc*3ffittth;fi!£©£5O 
[0 0 6 2 ] g=?\ Hi ©**«9t i-3TW^5*lfc« 

«iiiB«*/j«fc#wr* (E-i) c friw&tL 

T, WTO (1) ~ (3) ©30©7jj£tf#*.?>ft3 0 

( 1 ) H«li**fc*1*-t*ft£flS*3 x 3©ftM^c 

««fc-fS (HI 1 (a) #hs) o 

( 2 ) ffigffiWC fcfljgf *fi^»*Jtft tttttc 4 o© 

®(c # f j l , mrnmrn mtm^^n^ m®&>m 
mttZc ttowtitmvfrmmtt* mi 1 

(b) #BSJ „ 

( 3 ) MRmfsmtcHm- zmizo a o ©a^i: , 

«W*«0'MW*4:-r« (Hill (c) #F,s) o 

[0063] ^k, mat 1 0T*/jx*«t?ofeo»* 

■**i6« (E-2) o feO#*«i:LT{iWToaa 

•fesr B i (l' a' b - . l- U " v' m) ■vtomsM 



[0064] ccm g-^p-^gcsAigfcffla-rs 

T^4feo^^5f|^^i*VJ^Sl^c t?rfijffl LT«oran*fT 
5c -r**a*, ±T©»s? (±, T©'J^«) 0**1* 
ft©^fit>L<«#^»©¥£]#\ £i?©Mil? (WO 

*£t^fc£ (E-3) tc^©«««a|P,T^5t¥iJ 
^•r?. (E-4) . fLT, COffiPI*B«i«:9 0' f 

[0 0 6 5 ] fc*-£fafc-3^T©*tt*ilW&&ta-f 
t^2©<ta{c%Sc. »2K6^T, v(Xl) tt/jNffiJ«Xl 

[0 0 6 6] 
[»2] 

v(Xl) > v(X2) all XI, X2 or 

average (v (XI)) > average (v(X2)) or 

v(Xl) > average(v(X2)) all XI or 

average (v (XI)) > v(X2) all X2 

CCT\ XI : (x.y)=±<D^««, T©/j^ffi«TfeO, X 

2: (x.y)=«©/|N«*T?i6«o 



1 ] 

[0 0 6 7] ±, TO'WS^^tl^tlOfe©^ 

iswo* ti€nofe<D»*a t l < fi^a ©¥#)#& 
§ L £ o ffi Jil T x d © - - o © * 1 1 f c ilt ft -r i: § , 

b*)f £8S 3 © J: 5 ft 3 fcfcl^ 

T, v(Xl) ti/J%«WXlT<DfeO#*»*«LTV^*. 
[0 0 6 8] 

[83] I v(Xl) > CI all XI and/or 

v(X2) < C2 all X2 ) 
or 

I average (v (XI)) > C3 and/or 
average (v(X2)) < C4 } 

CCT% XI : (x,y)=±<0/h«l*, TO/h*«T*»), X 
2: (x,y)=*0/h««TS6*. 

[0 0 6 9] El 1 0fcSLfcSB5«D*»ff!lte:«ttltf, P, 

©gpfs c* t © l t sarr 5 ft 46 is it w« t > 

[0 0 7 0] Ell 0t^Lft^5©Slffil?UT-«# 

m<D¥-i%&V Mi%ffiffl&£ik (DW-i^&lcMt % fe©if© 
«ii, ^iRii:*ffli^^ffit,*« (Hi 2#P,?J „ cc 

[0 0 7 1 ] £f, £5Q«ttHH£Htt£ffi£*fflM£ 

feffl^ffli^i:<ki\ 
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(54) [Title of the Invention] 
FACE EXTRACTING METHOD 



15 (57) [Abstract] 
[Object] 

To provide a face extracting method for 
extracting totally automatically a person's face from a 
color original image of negative film or positive film 

20 without assistance and at high precision. 
[Constitution] 

The person's face is extracted from the color 
original image by deciding a face candidate area 
corresponding to the shape of person's face and 

25 deciding a face area from the feature amounts within 
the face candidate area. 
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[Claims for the Patent] 
[Claim 1] 

A face extracting method for extracting a 
person's face from an image, characterized by 
5 comprising deciding a face candidate area corresponding 
to the shape of the person's face, and deciding a face 
area from the feature amounts within said face 
candidate area. 
[Claim 2] 

10 A face extracting method characterized by 

comprising detecting a face candidate area by 
extracting the contour of a person's face from an image. 
[Claim 3] 

A face extracting method for extracting a 
15 person's face from an image, characterized by 

comprising calculating the matching degrees between a 
plurality of templates in the shape of face and said 
image, selecting a template with the highest matching 
degree, and making an area within the selected template 
20 a face candidate area, if the highest matching degree 
is more than or equal to a preset threshold value. 
[Claim 4] 

The face extracting method according to any one 
of claims 1 to 3, characterized by comprising acquiring 
25 a one-dimensional histogram of a parameter representing 
the color within said face candidate area, and deciding 
the face area from the shape of said histogram. 

- 2 - 
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[Claim 5] 

The face extracting method according to any one 
of claims 1 to 3, characterized by comprising acquiring 
a two-dimensional histogram of a parameter representing 
5 the color within said face candidate area, and deciding 
the face area from the shape of said histogram. 
[Claim 6] 

The face extracting method according to claim 4, 
characterized by comprising using a histogram of 
10 lightness as the one-dimensional histogram of said 
parameter representing the color. 
[Claim 7] 

The face extracting method according to any one 
of claims 1 to 3, characterized by comprising deciding 
15 the face area from a distribution amount of the 

parameter representing the color within said face 
candidate area. 
[Claim 8] 

The face extracting method according to claim 7, 
20 characterized by comprising using the area of a 

covariance ellipse of hue and lightness in the two- 
dimensional space as the distribution amount of said 
color . 
[Claim 9] 

25 The face extracting method according to any one 

of claims 1 to 3, characterized by comprising deciding 
the face area from a spatial frequency response of said 



- 3 - 



English translation of Japanese Patent Application Laid-Open 
No. 08-063597 

face candidate area. 
[Claim 10] 

The face extracting method according to any one 
of claims 1 to 3, characterized by comprising dividing 
5 said face candidate area into small areas, obtaining 
the feature amounts for each small area, and deciding 
the face area using the feature amount distribution 
within said face candidate area. 
[Claim 11] 

10 The face extracting method according to claim 10, 

characterized in that the eye, mouth and cheek are 
separated by division into said small areas. 
[Claim 12] 

The face extracting method according to claim 10 
15 or 11, characterized by comprising using a distribution 
amount of color in the small area as said feature 
amount . 
[Claim 13] 

The face extracting method according to claim 12, 
20 characterized by comprising using the area of a 

covariance ellipse of hue and lightness in the two- 
dimensional space as the distribution amount of said 
color . 
[Claim 14] 

25 The face extracting method according to claim 10 

or 11, characterized by comprising using a color 
difference amount and direction of the average color in 

- 4 - 
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the small area from the average color in the face 
candidate area as said feature amount. 
[Claim 15] 

The face extracting method according to claim 14, 
5 characterized by comprising using a differential vector 
between two average colors of hue and lightness in the 
two-dimensional space as the color difference amount 
and direction of the average color in the small area 
from the average color in the face candidate area. 
10 [Claim 16] 

The face extracting method according to claim 10 
or 11, characterized by comprising using a spatial 
frequency property as said feature amount. 
[Claim 17] 

15 The face extracting method according to claim 16, 

characterized by comprising using a spatial frequency 
response of hue as the spatial frequency property in 
said small area. 
[Claim 18] 

20 The face extracting method according to any one 

of claims 1 to 3, characterized by comprising 
extracting the eyes within said face candidate area and 
deciding the face area. 
[Claim 19] 

25 The face extracting method according to any one 

of claims 1 to 3, characterized by comprising 
extracting the eye area within said face candidate area, 

- 5 - 
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extracting the mouth from the positional information of 
eyes, and deciding the face area. 
[Claim 20] 

The face extracting method according to any one 
5 of claims 1 to 3, characterized by comprising 

extracting the eye area within said face candidate area, 
extracting the hair from the positional information of 
eyes, and deciding the face area. 
[Claim 21] 

10 The face extracting method according to any one 

of claims 1 to 3, characterized by comprising 
extracting the eye area within said face candidate area, 
extracting the neck from the positional information of 
eyes, and deciding the face area. 

15 [Claim 22] 

The face extracting method according to any one 
of claims 1 to 3, characterized by comprising expanding 
the face candidate area after deciding said face 
candidate area, extracting the hair area, and deciding 

20 the face area. 
[Claim 23] 

The face extracting method according to any one 
of claims 1 to 3, characterized by comprising expanding 
the face candidate area after deciding said face 
25 candidate area, extracting the neck area, and deciding 
the face area. 

- 6 - 
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[Claim 24] 

The face extracting method according to any one 
of claims 1 to 3, characterized by comprising deciding 
the face area using a matched filter within said face 
5 candidate area. 
[Claim 25] 

The face extracting method according to any one 
of claims 1 to 3, characterized by comprising acquiring 
the feature amount within said face candidate area, 
10 inputting said feature amount into a neural network and 
deciding the face area. 
[Claim 26] 

The face extracting method according to claim 25, 
characterized by comprising using a feature amount 
15 representing the eyes, a feature amount representing 

the mouth, a feature amount representing the hair, and 
a feature amount representing the neck as said feature 
amounts . 
[Claim 27] 

20 A face recognition device that can be finely 

adjusted by inputting the new teacher data in said 
neural network after shipping an apparatus 
incorporating the face extracting method according to 
claim 25 or 26. 

25 [Claim 28] 

A color printer that decides the exposure value 
to the copying material based on data of person's face 

- 7 - 
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extracted by the face extracting method according to 
any one of claims 1 to 26. 
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[Detailed Description of the Invention] 
[0001] 

[Industrial Application Field] 

The present invention relates to a face 
5 extracting method for extracting a part of face from a 
color original image such as negative film or positive 
film on which a person is photographed by a camera. 
[0002] 

[Conventional Art] 

10 In copying a color original image onto the 

copying material, such as printing a negative film 
taken by a camera on the positive, it is important to 
copy with an appropriate exposure value, and 
particularly in the color photograph of taking a 

15 picture of the person, if the image is printed to make 
the right color of person's face, the person looking at 
the photograph generally has a good feeling, whereby 
the quality of photograph is enhanced. 
[0003] 

20 In photography, if the photographing conditions 

are always constant, the printing is made with an 
appropriate exposure value according to the 
photographing conditions. In practice, the original 
image taken in backlight or the original image taken 

25 using a strobe may be mixed in one roll of film. 

Therefore, to produce the high quality photos, it is 
preferable to change the exposure value of printing for 

- 9 - 
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every original image. To decide the exposure value of 
printing, it is convenient to pay attention to the 
color of person's face in the photograph of a person. 
Since it is known in advance that the color of face is 
5 flesh color, the exposure value can be decided so that 
the color of person's face in the photograph after 
printing may be flesh color. 
[0004] 

The methods for deciding the exposure value of 
10 printing on the basis of the color of person's face in 
this manner have been conventionally known, and are 
employed to automate a part of the developing and 
printing operation for the color film. 
[0005] 

15 For example, a method has been already known in 

which the operator specifies a face area in the 
original image of the color film with a light pen, the 
density data of the person's face is extracted, and the 
exposure value is decided to print the color of face 

20 rightly based on the extracted density data. 
[0006] 

Also, a method for extracting the person's face 
in the original image has been known in which the flesh 
color data is extracted from the original image, and a 
25 cluster of photometric points judged as the flesh color 
range is made the face (Japanese Patent Application 
Laid-Open No. 52-156624, Japanese Patent Application 

- 10 - 
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Laid-Open No. 53-145621, Japanese Patent Application 
Laid-Open No. 53-145622) . This method involves 
dividing the color original image into a number of 
photometric points, decomposing each photometric point 
5 into three colors of R (red), G (green) and B (blue) 
for photometric measurements, judging whether or not 
the color of each photometric point computed from 
photometric data is within the flesh color range, and 
making the cluster (group) of photometric points judged 
10 as the flesh color range the density data of face. 
[0007] 

Further, in Japanese Patent Application Laid-Open 
No. 4-34 6333, a method of converting the photometric 
data into hue value (H) and saturation value (S), 

15 creating a two-dimensional histogram for H and S, 
dividing this histogram into unimodal mountains, 
judging which of the divided mountains each pixel of 
the original image belongs to, extracting the face 
candidate area by and dividing the pixels, and judging 

20 whether or not the image is the person based on the 
contour and internal structure of the face candidate 
area was disclosed. 
[0008] 

[Problems to be Solved by the Invention] 
25 With the above method in which the operator 

specifies the face area with a light-pen, there is a 
problem that it takes a lot of time to perform the 
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printing operation, because the face area in the color 
original image can be extracted without fail, but the 
operator has to specify the face area for every image 
with the light-pen. Also, with this method, the total 
5 unmanned operation (automation) without interposition 
of the operator is impossible. 
[0009] 

On the other hand, with the method of extracting 
flesh color data from the original image, and making a 

10 cluster of photometric points judged as the flesh color 
range the face, there is a problem that a region having 
flesh color or approximate flesh color such as ground, 
trunk of tree, clothes other than the face may be 
extracted as the density data of face, and the 

15 precision is poor. Also, there is a problem that the 

face may not be extracted depending on the film species 

or the light source. 

[0010] 

And with the method as disclosed in Japanese 
20 Patent Application Laid-Open No. 4-346333, when the 

face and hand or the face and face are contacted, the 
shape of flesh color area becomes complex, whereby 
there is a problem that the face alone can not be 
detected . 
25 [0011] 

The invention has been achieved in the light of 
the above-mentioned problems, and it is an object of 
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the invention to provide a face extracting method for 
extracting totally automatically a person's face from a 
color original image such as negative film or positive 
film without assistance and precisely. 
5 [0012] 

[Means for Solving the Problems] 

In order to accomplish the above object, the 
present invention provides a face extracting method for 
extracting a person's face from an image by comprising 
10 deciding a face candidate area corresponding to the 
shape of the person's face, and deciding a face area 
from the feature amounts within the face candidate area. 
[0013] 

Also, another method comprises detecting a face 
15 candidate area by extracting the contour of a person's 
face from an image. 
[0014] 

Further, another face extracting method for 
extracting a person's face from an image comprises 

20 preparing a plurality of templates having the various 
shapes of face, computing the matching degrees between 
the templates and the image, selecting a template with 
the highest matching degree, and making an area within 
the selected template a face candidate area, if the 

25 highest matching degree is more than or equal to a 
preset threshold value. 
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[0015] 
[Operation] 

With the above method of the invention, the 
person's face is extracted from the color original 
5 image by deciding the face candidate area corresponding 
to the shape of person's face and deciding the face 
area from the feature amounts within the face candidate 
area . 
[0016] 

10 Also, with another method, the person's face is 

extracted from the color original image by extracting 
the contour of the person's face from the image and 
detecting the face candidate area. 
[0017] 

15 Further, with another method, the person's face 

is extracted from the color original image by preparing 
a plurality of templates having the various shapes of 
face, computing the matching degrees between the 
templates and the image, selecting a template with the 

20 highest matching degree, and making an area within the 
selected template a face candidate area, if the highest 
matching degree is more than or equal to a preset 
threshold value. 
[0018] 

25 [Embodiments] 

The present invention will be described below 
with reference to the drawings. 
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[0019] 

Figure 1 is a block diagram of a face area 
extracting device using a face extracting method 
according to the invention. 
5 [0020] 

A film 1 is one on which a color original image 
is recorded. It does not matter whether the film is a 
negative film or a positive film. When the flesh color 
is extracted from the positive image, the flesh color 

10 may be directly extracted from the photometric value of 
positive. When the flesh color is extracted from the 
negative image, the flesh color may be extracted by 
converting the photometric value of negative into 
positive, or may be directly extracted from the 

15 photometric value of negative. A scanner 2 optically 

scans the color original image of the film 1, and makes 
the color separation to produce the B (blue) , G (green) 
and R (red) values of each pixel. The BGR values are 
amplified by an amplifier 3, converted into digital 

20 data by an A/D converter 4, and inputted into a CPU 5. 
The CPU 5 performs each process for face extraction as 
will be described later. 
[0021] 

Figure 2 is a flowchart of a face extracting 
25 method according to a first embodiment of the invention. 
[0022] 

First of all, the feature amounts of color such 
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as lightness, hue, saturation, chromaticity (B-R) and 
(G-R) are obtained from the BGR values of each pixel 
produced by the scanner 2 (A-l) . If these feature 
amounts of color are within the preset ranges, it is 
5 determined that the object pixel is flesh color (A-2). 
Of course, when the flesh color is extracted from the 
BGR values of negative image, and when the flesh color 
is extracted from the BGR values of positive image, the 
preset ranges for the feature amounts are different. 
10 Also, the flesh color or not may be determined, with 
the obtained feature amounts of color as the input 
values into a neural network. 
[0023] 

Next, the edge extraction for the image composed 
15 of pixels determined as the flesh color is performed 
(A-3) . An edge extraction method involves taking an 
average of lightness over the peripheral 8 pixels 
around an object pixel, making the object pixel an edge 
pixel, if a difference between the average value and 
20 the lightness of the object pixel is larger than a 

predetermined value, and binarizing the object pixel 
depending on whether the object pixel is the edge pixel 
or not, for example. Herein, the image obtained in 
this manner is called an edge image. 
25 [0024] 

At step (A-4), firstly, a plurality of face 
templates (see Figure 3) of ellipse or circle which are 
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different in the size and ratio of major axis/minor 
axis (la/lb) are created. These face templates may be 
created in advance, and stored in a storage unit 6. 
The face template is binarized depending on whether or 
5 not the contour is ellipse or circle. Since the actual 
face contour is not a correct ellipse or circle, the 
contour of the face template may have a width of 
several pixels, preferably, 2 to 3 pixels to increase 
the matching degree with the actual face contour. 
10 [0025] 

At step (A-4), the matching degree between the 
edge image and the face template is obtained. To 
obtain the matching degree, an existent method is used. 
For example, the matching degree m(u,v) is obtained in 
15 accordance with Formula 1. 
[0026] 
[Formula 1] 



, . // s ((f(x+u.y+v)-f>(t(x,y}-f))dxdy 
m(u,v)=-T . _ _ — 

1 ///s(f(x+u,y+v)-f) 2 dJcdy-// s (t(x,y)-t') 2 dxdy 



In the Formula 1, f denotes the object image, t 
20 denotes the face template, and S denotes the value 
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range of t (x, y) . Also, f and t 1 denote the average 

of f(x+u, y+v) and t(x, y) in S. 

[0027] 

With this method, the matching is made using 
5 several kinds of face templates, and the most matched 
template is obtained for the object pixel (A-4) . If 
the matching degree is more than or equal to a preset 
threshold value, the area surrounded by the most 
matched face template around the object pixel is 
10 determined as the face candidate area (A-6) . 
[0028] 

Also, to reduce the number of operations, firstly, 
as a rough scan, the matching degree with the face 
template of fixed size may be obtained for every pixel, 

15 or every several pixels, and the template of different 
size may be fitted to only the object pixel in which 
the matching degree is more than or equal to a 
predetermined value, thereby deciding the optimal 
candidate area. 

20 [0029] 

The first embodiment as shown in Figure 2 has the 
effect that the face simplex can be extracted correctly 
even from the image where the face and hand, or the 
face and face are contacted, for example, because the 
25 face contour is extracted. 
[0030] 

Also, if the range of color regarded as the flesh 
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color is set wider in extracting the flesh color area, 
it is possible to cover a difference in the kind of 
film or the light source. 
[0031] 

5 Further, if the range of flesh color is set for 

each of the negative image and the positive image, the 
face can be extracted from both the negative image and 
the positive image. 
[0032] 

10 While in the first embodiment as shown in Figure 

2, the matching degree with the face template is 
obtained after the flesh color area is extracted, the 
invention is not limited thereto, but the entire object 
image may be converted into the lightness image 

15 (without extracting the flesh color) , and then the face 
candidate area may be decided by the same method as in 
the first embodiment. 
[0033] 

Also, while in the first embodiment as shown in 
20 Figure 2, the matching between the image binarized 

depending on edge or not and the face template of edge 
is made, the invention is not limited thereto, but 
after extracting the flesh color area, the matching 
between the image binarized depending on flesh color or 
25 not and the face template binarized depending on flesh 
color or not may be made. 
[0034] 
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By the way, it suffices that a portion determined 
as the face candidate area is regarded as the face in 
the first embodiment, but any other portion than the 
face may be determined as the face candidate area 
5 depending on the image. Thus, a method for extracting 
an actual portion of the face surely by further 
narrowing down a plurality of portions determined as 
the face candidate area in the first embodiment will be 
described below. 
10 [0035] 

Figure 4 is a flowchart of a face extracting 
method according to a second embodiment of the 
invention . 
[0036] 

15 First of all, an area with eyes (eye candidate 

area) within the face candidate area determined in the 
first embodiment is estimated and set (B-l) . At this 
time, the area is estimated in the case where the image 
is longitudinal (see Figure 5A) and the case where the 

20 image is transversal (see Figure 5B) . In this 

estimation method, for example, when the image is 
directed longitudinally, the upper half and the lower 
half of the face candidate area are set as the eye 
candidate area, while when the image is directed 

25 transversally, the right half and the left half of the 
face candidate area are set as the eye candidate area. 
That is, the upper half, the lower half, the right half 

- 20 - 



English translation of Japanese Patent Application Laid-Open 
No. 08-063597 

and the left half of the face candidate area are set as 
the eye candidate area. With this method for setting 
the eye candidate area, the narrower parts of the face 
candidate area may be set. For example, when the image 
5 is directed longitudinally, the range where the height 
is from 0.5 to 0.8 may be set as the eye candidate area, 
when the height of the lower end of the face candidate 
area is 0 and the height of the upper end is 1, as 
shown in Figure 6. 
10 [0037] 

Next, a histogram for lightness is created within 
each eye candidate area (B-2), as shown in Figure 7. 
Herein, an instance where the lightness is represented 
in 8 bits will be described below (lightness 0:0, 

15 lightness 100:255). Firstly, the lightness area (0 to 
255) is evenly divided into eight sub-areas, and the 
appearance frequency within each lightness sub-area is 
obtained. Since the portion with black eye has a low 
lightness, the histogram has the peaks in the black eye 

20 portion of lightness and the flesh color portion of 
lightness (see Figure 7). Though the histogram of 
lightness is created herein, the invention is not 
limited thereto, but the saturation, hue or 
chromaticity may be used instead of the lightness. 

25 [0038] 

By the way, one of the shape recognition methods 
for the histogram created in this manner is as follows. 
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First of all, the highest peak (first peak) in the 
lightness range (e.g., 96 or more with 8 bits) 
considered as the flesh color area is searched. Next, 
the highest peak (second peak) in the lightness range 
5 lower than a predetermined value is searched. This 

predetermined value is empirically obtained by metering 
several samples of eyes using a measuring instrument. 
The value of (frequency of second peak/frequency of 
first peak) is calculated and made the feature amount 
10 (B-3). 
[0039] 

If this feature amount is within a predetermined 
range (B-4), it is judged that the shape of this 
histogram represents the face. Since the predetermined 
15 range at step (B-4) is different depending on the size 
of eye candidate area, the optimal range must be 
obtained in each case. 
[0040] 

In this manner, a judgement is made for each eye 
20 candidate area, in which if there is any area judged as 
the eye, it is determined that the face candidate area 
is the face (B-4), or if there is no area judged as the 
eye, it is determined that the face candidate area is 
not the face. 
25 [0041] 

Though in the second embodiment as shown in 
Figure 4, the eye candidate area is set at step (B-l) , 
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and the histogram of lightness within the eye candidate 
area is created at step (B-2), the invention is not 
limited thereto, but the eye candidate area may not be 
set, and the histogram of lightness may be created over 
5 the entire face candidate area to determine whether or 
not the face candidate area is the face, for example. 
[0042] 

With the second embodiment as shown in Figure 4, 
it is possible to handle a difference in the direction 
10 of image by setting a plurality of eye setting areas 
within the face candidate area. 
[0043] 

In other words, there is the effect that the 
direction of image can be automatically determined by 
15 this method. 
[0044] 

Though in the second embodiment as shown in 
Figure 4, the one-dimensional histogram of lightness is 
created, the invention is not limited thereto, but the 

20 two-dimensional histogram with lightness and hue along 
the axes may be created to make a determination by the 
same method as in the second embodiment, with the peak 
in the area representing the flesh color as the first 
peak, and the peak in the area representing the black 

25 of eye as the second peak, or the two-dimensional 
histogram of lightness and saturation or the two- 
dimensional histogram of hue and saturation may be 
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employed . 
[0045] 

Figure 8 is a flowchart of a face extracting 
method according to a third embodiment of the invention. 
5 [0046] 

First of all, the image within the face candidate 
area determined in the first embodiment is converted 
into lightness (C-l) . Though in this embodiment, the 
image within the face candidate area is converted into 
10 lightness, the image may be converted into chromaticity , 
hue or saturation image, instead of lightness image. 
[0047] 

Next, the size of the face candidate area is 
normalized by scaling up or down the face candidate 
15 area to conform with the reference size (C-2). And the 
two-dimensional Fourier transform is made for the 
normalized face candidate area (C-3) . 
[0048] 

Herein, as a result of this Fourier transform, 
20 the power spectrum is normalized by the greatest peak 
value. And several patterns (hereinafter referred to 
as "reference data of face") actually representing the 
face are prepared by similarly making the two- 
dimensional Fourier transform, and prestored in the 
25 storage unit 6. The matching between the Fourier 

transform result for the face candidate area and the 
Fourier transform result for the reference data of face 
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is made, whereby the value with the highest matching 
degree is made the feature amount (C-4). 
[0049] 

And if this feature amount is more than or equal 
5 to the threshold value (C-5) , it is determined that 
this face candidate area is the face (C-6) . 
[0050] 

By the way, though in the third embodiment as 
shown in Figure 8, the two-dimensional Fourier 

10 transform is made at step (C-3) , the invention is not 
limited thereto, but the one-dimensional Fourier 
transform may be made. In this case, it is required to 
prepare the patterns of full-face and face in portrait 
as the reference data of face in the combinatorial 

15 conditions where the direction of image is up, down, 
right and left. 
[0051] 

With the third embodiment as shown in Figure 8, 
there is the effect that even when the portion (eye or 
20 mouth) or position of face is dislocated due to 

incorrect extraction of the face candidate area, the 
high extraction ratio of face can be attained without 
being affected by the dislocation. 
[0052] 

25 While in the third embodiment as shown in Figure 

8, the two-dimensional Fourier transform is made, the 
two-dimensional Fourier transform may not be made in 
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this invention, but other methods according to the 
frequency characteristics may be employed. For example, 
the following method may be adopted. 
[0053] 

5 First of all, supposing that the abscissa and the 

ordinate for the image are x-axis and y-axis, the 
projection of lightness onto the x-axis is performed in 
which the value of lightness at the position x 0 on the 
x-axis is added along the y-axis and made the value of 

10 x 0 . Similarly, the projection of lightness onto the y- 
axis is performed. And for each of the projection onto 
the x-axis and the projection onto the y-axis, the 
projected lightness is normalized by each peak value. 
The matching between this data and the reference data 

15 actually obtained from the face image in the same 

manner is made, and if the matching degree is more than 
the threshold value, the face candidate area is judged 
as the face. Of course, the lightness image may not be 
employed, but it is only required that the feature 

20 amount representing the color such as hue, chromaticity 
or saturation is projected onto the x-axis or y-axis. 
[0054] 

In this case, the calculation time is shorter 
than the two-dimensional Fourier transform, whereby 
25 there is the effect that the processing can be 
performed at high speed. 
[0055] 
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Figure 9 is a flowchart of a face extracting 
method according to a fourth embodiment of the 
invention . 
[0056] 

5 First of all, the hue and lightness of each pixel 

within the face candidate area determined in the first 
embodiment are obtained (D-l), and the area of a 
covariance ellipse of hue and lightness in the two- 
dimensional space is obtained and made the feature 
10 amount (D-2) . 
[0057] 

If the area of the covariance ellipse obtained at 
step (D-2) is within a predetermined range (D-3) , it is 
determined that the face candidate area is the face (D- 
15 4) . 

[0058] 

Instead of the covariance ellipse, the volume of 
variance ellipsoid in the color space (L'a'b', L'u'v', 
etc.), variance of hue, variance of lightness, and 
20 variance of saturation may be employed as the feature 
amounts. Or at least two or more of the above feature 
amounts may be combined to make the judgment, whereby 
the recognition ratio is increased. 
[0059] 

25 With the fourth embodiment as shown in Figure 9, 

the feature extraction can be made without regard to 
the direction of image. 
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[0060] 

Also, if the feature amounts obtained in the 
second embodiment as shown in Figure 4 may be combined 
to make the judgement, the recognition ratio is further 
5 increased. 
[0061] 

Figure 10 is a flowchart of a face extracting 
method according to a fifth embodiment of the invention. 
[0062] 

10 First of all, the face candidate area determined 

in the first embodiment is divided into small areas (E- 
1). The following three dividing methods (1) to (3) 
are provided. 

(1) A rectangle circumscribing the face candidate 
15 area is divided into 3x3 rectangles, and the areas 

overlapping the face candidate area are made the small 
areas. Three small areas at the upper stage are upper 
small areas, the small area in the center at the lower 
stage is the lower small area, and the small areas at 
20 both ends at the middle and lower stages are lateral 
small areas (see Figure 11A) . 

(2) A rectangle circumscribing the face candidate 
area is diagonally divided into four triangles, and the 
areas overlapping the face candidate area are made the 

25 small areas. The left and right areas are made the 
lateral small areas (see Figure 11B) . 

(3) The areas of the candidate area overlapping 
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near four sides of a rectangle circumscribing the face 
candidate area are made the small areas. The left and 
right areas are made the lateral small areas (see 
Figure 11C) . 
5 [0063] 

Next, the distribution amount of color in each 
small area is obtained as the feature amount (E-2). 
The distribution amount of color has the following 
kinds . 

10 ■ Area of covariance ellipse of hue and lightness 

in the two-dimensional space 

• Volume of variance ellipsoid in the color space 
(L'a'b' , L'u' v' , etc. ) 

• Variance of hue 

15 • Variance of lightness 

• Variance of saturation 

[0064] 

Herein, the face recognition is made using the 
fact that the distribution amount of color is large in 

20 the small area corresponding to the eye, mouth or 

hairline, and the distribution amount of color is small 
in the small area corresponding to the cheek. That is, 
when the distribution amount of each of the upper and 
lower areas (upper and lower small areas) or the 

25 average of distribution amount is larger than the 

distribution amount of each of the left and right areas 
(lateral small areas) or the average of distribution 
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amount (E-3), it is determined that the face candidate 
area is the face (E-4). And this operation is 
performed by rotating the image by every 90°. 
[0065] 

5 Representing the conditions in one direction in 

the expression, Formula 2 is obtained. In Formula 2, 

v(Xl) denotes the distribution amount of color in the 

small area XI. 

[0066] 
10 [Formula 2] 

v(Xl)> v(X2) all XI, X2 or 

average (v(Xl))> average (v(X2)) or 

v(Xl)> average (v(X2)) all XI or 

average (v(Xl))> v(X2) all X2 
15 Where XI: (x, y)= upper small area, lower small 

area, X2 : (x, y)= lateral small area. 

[0067] 

There is another method for determining that the 
face candidate area is face if both or one of the 

20 condition where the distribution amount of color in 

each of the upper and lower small areas or the average 
of distribution amount is more than or equal to a 
certain threshold value, and the condition where the 
distribution amount of color in each of the lateral 

25 small areas or the average of distribution amount is 
less than or equal to a certain threshold value is 
satisfied. In this case, the operation is performed by 
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rotating the image by every 90°. Representing the 
condition in one direction in the expression, Formula 3 
is obtained. In the Formula 3, v(Xl) denotes the 
distribution amount of color in the small area XI. 
5 [0068] 

[Formula 3] 

v {XI; > CI all XI and/or 
v(X2)< C2 all X2 
or 

10 Gaverage (v(Xl))> C3 and/or 
average (v2 (X2) ) < C4 

Where XI: (x, y)= upper small area, lower small 
area, X2 : (x, y)= lateral small area. 
[0069] 

15 With the fifth embodiment as shown in Figure 10, 

there is the effect that the recognition ratio for 
recognizing the image is high by making effective use 
of the features in each portion of the face. 
[0070] 

20 While in the fifth embodiment as shown in Figure 

10, the distribution amount of color is used as the 
feature amount, the feature amount may be the color 
difference amount and direction of the average color in 
small area from the average color over the entire face 

25 candidate area (see Figure 12) . This method will be 
described below. 
[0071] 

- 31 - 



English translation of Japanese Patent Application Laid-Open 
No. 08-063597 

First of all, like the fifth embodiment, the face 
candidate area is divided into small areas, and the 
color difference amount and direction of the average 
color in small area from the average color over the 
5 entire face candidate area are obtained. The lightness 
in judging the eye or hair, and the hue in judging the 
mouth may be employed. 
[0072] 

The difference has the following kinds. 
10 ■ Differential vector of two average colors of 

hue and lightness in the two-dimensional space 

■ Differential vector of two average colors in 
the color space (L'a'b', L'u'v', etc.) 

• Difference in the average of hue 

15 ■ Difference in the average of lightness 

■ Difference in the average of saturation 

[0073] 

The following kinds of average computation method 
are considered. 
20 • Even average within small area 

• Average weighted centrally in small area 

[0074] 

Herein, when the direction of color difference 
between the average color in small area and the average 
25 color is the direction of black, red and black in 

candidate area in the small area corresponding to the 
eye, mouth and hairline, the candidate area is 
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recognized as the face. Specifically, when all or some 
of the following three conditions are satisfied, the 
face candidate area is determined as the face. 

(1) The color difference direction in the average 
5 color between the upper small area and the face 

candidate area or the color difference average 
direction is the direction where the lightness is 
smaller . 

(2) The color difference direction in the average 
10 color between the lower small area and the face 

candidate area or the color difference average 
direction is the direction from the flesh color to red. 

(3) The color difference amount in the average 
color between the lateral small area and the face 

15 candidate area or the average of color difference 
amount is smaller than a certain threshold value. 
[0075] 

And this operation is performed by rotating the 
image by every 90°. 
20 [0076] 

While in the fifth embodiment as shown in Figure 
10, the distribution amount of color is used as the 
feature amount, the feature amount may be the spatial 
frequency property (see Figure 13) . Herein, this 
25 method will be described below. 
[0077] 

First of all, like the fifth embodiment, the face 
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candidate area is divided into small areas, and the 
value of spatial frequency property is obtained as the 
feature amount. 
[0078] 

5 The following kinds of spatial frequency object 

are conceived. 

• Hue 

■ Lightness 

• Saturation 

10 [0079] 

And the feature amount can be obtained in the 
following ways. 

• The two-dimensional Fourier transform is 
performed, and a ratio of the integral value of 

15 response within the high frequency area to the integral 
value of response over the entire spatial frequency 
area is the feature amount. 

• The two-dimensional Fourier transform is 
performed to obtain a power spectrum, and the height of 

20 peak in the high frequency area is the feature amount. 

• The histogram of the average value of 
differences of concerned pixel from the all adjacent 
pixels is obtained, and a ratio of pixels in which the 
average value of differences is more than or equal to a 

25 predetermined value is the feature amount. 
[0080] 

Herein, when the feature amount in small area 
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corresponding to the eye, mouth or hairline is large, 
the face candidate area is determined as the face. 
That is, when the feature amount in lateral small area 
is larger than the feature amount in the upper and 
5 lower small areas, the face candidate area is 

determined as the face. After all, this operation is 
performed by rotating the image by every 90°. 
[0081] 

Figure 14 is a flowchart of a face extracting 
10 method according to a sixth embodiment of the invention. 
[0082] 

First of all, the size of eye is estimated, based 
on the size of the face candidate area determined in 
the first embodiment, and the template of eyes as shown 
15 in Figure 15A and the template of eyes with eyeglasses 
as shown in Figure 15B are created (F-l). 
[0083] 

And an object image within the face candidate 
area is binarized depending on whether or not the image 

20 represents the eyes (F-2). Though it does not matter 
if the black pixel representing the eyes is detected 
from the color information such as lightness, 
saturation and hue as usual, the following method is 
more preferable. 

25 (1) Firstly, the average of lightness of flesh 

color pixels within the face candidate area is obtained. 
(2) If the difference from this average value is 
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more than or equal to the threshold value (of course, 
the lightness of pixels representing the eyes is 
smaller) , it is determined that the black pixels 
represent the eyes. 
5 [0084] 

In this manner, it is possible to cover 
differences in the luminosity of black pixels of the 
eyes due to differences in the luminosity of the face. 
[0085] 

10 The eye template is scanned in the vertical 

direction and the horizontal direction over the face 
candidate area binarized at step (F-2) (see Figure 15A) , 
whereby the highest matching degree is obtained and 
made the feature amount (F-3) . 

15 [0086] 

And if this feature amount is more than or equal 
to the threshold value (F-4), the face candidate area 
is determined as the face (F-5) . 
[0087] 

20 With the sixth embodiment as shown in Figure 14, 

there is the effect that it is possible to determine 
whether or not the object wears the eyeglasses, and 
determine the direction of image. 
[0088] 

25 Figure 16 is a flowchart of a face extracting 

method according to a seventh embodiment of the 
invention . 
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[0089] 

In this embodiment, after the process of the 
sixth embodiment as shown in Figure 14 is performed, a 
judgement is made for the mouth in addition to the eyes. 
5 Accordingly, steps (G-l) to (G-4) are the same as in 
the flowchart of Figure 14, and not described here. 
[0090] 

At step (G-5), the mouth position is estimated 
from the position information of the eyes. For example, 

10 if the positions of both eyes are in the upper half of 
the face, it is estimated that the mouth position is in 
the lower half of the face. Also, if the positions of 
both eyes are in the right half of the face, it is 
estimated that the position of mouth is in the left 

15 half of the face. Also, if the position of mouth is 
estimated from the distance from the middle point of 
the line connecting both eyes as usually performed, the 
higher detection precision is obtained. 
[0091] 

20 Next, it is determined whether or not the mouth 

exists in the mouth candidate area from the color 
information. Specifically, the BGR signals are 
represented in 8 bits (0 to 255) . G-R is computed over 
the entire face candidate area, whereby its average 

25 value is obtained, for example. Similarly, G-R is 

calculated for the mouth candidate area. A difference 
between this computed value and the average value 
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previously obtained is obtained, and if this difference 
is more than or equal to the threshold value, the count 
of mouth pixel is incremented. 
[0092] 

5 And the ratio of mouth pixels (count number of 

mouth pixels/total number of pixels within the face 
candidate area) is obtained, and made the second 
feature amount (G-6) . 
[0093] 

10 If this second feature amount is within a 

predetermined range (G-7), the mouth candidate area is 
determined as the mouth, and the face candidate area is 
determined as the face (G-8) . Each threshold value may 
be empirically obtained using a measuring instrument. 

15 Instead of G-R, the other color information, for 

example, lightness, hue, saturation and chromaticity , 
may be employed. 
[0094] 

With the seventh embodiment as shown in Figure 16, 
20 it is known in which portion of the face area the 
position of mouth is located, and the direction of 
image can be determined. Also, there is the effect 
that the face extraction can be made at higher 
precision . 
25 [0095] 

While in the seventh embodiment as shown in 
Figure 16, it is determined whether or not the face 
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candidate area is the face by detecting the mouth in 
addition to the eyes, the invention is not limited 
thereto, but the hair may be detected in addition to 
the eyes to determine whether or not the candidate area 
5 is the face, for example. This method will be 
specifically described below. 
[0096] 

If the eye position is decided, the hair position 
is estimated from the eye position information. 

10 Specifically, if both eyes are in the upper half of the 
face candidate area as shown in Figure 17, the boundary 
area above the line connecting both eyes is estimated 
as the hair candidate area, whereby the black pixels 
indicating the hair within this area are counted. Also, 

15 if both eyes are in the right half of the face 

candidate area, the boundary area to the right of the 
line connecting both eyes is estimated as the hair 
candidate area. 
[0097] 

20 And if a ratio of hair pixels (number of black 

pixels/area of hair candidate area) is more than or 
equal to the threshold value, it is determined that 
there is the hair, and this face candidate area is 
determined as the face. Herein, if the size of the 

25 face candidate area is expanded by several pixels 

before detecting the hair, the detection precision is 
increased . 
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[0098] 

In this case, there is the effect that the 
direction of the image can be determined depending on 
where the hair area is located in the upper half, lower 
5 half, left half, or right half of the face. 
[0099] 

While in the seventh embodiment as shown in 
Figure 16, it is determined whether or not the 
candidate area is the face by detecting the mouth in 

10 addition to the eyes, the invention is not limited 

thereto, but the neck may be detected in addition to 
the eyes to determine whether or not the candidate area 
is the face. This method will be specifically 
described below. 

15 [0100] 

If the eye position is decided, the neck position 
is estimated from the eye position information. For 
example, if both eyes are in the upper half of the face 
candidate area, as shown in Figure 17, the lower one- 
20 third part is made the neck candidate area. A wider 
area such as the lower half part may be employed. 
[0101] 

Next, the size of the face candidate area is 
expanded by 3 or 4 pixels, and pixels of flesh color 
25 representing the neck are counted in the area contact 
with the neck candidate area. If the ratio of neck 
pixels (count number/area of face candidate area) is 
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more than or equal to the threshold value, the neck is 
decided, and this face candidate area is determined as 
the face. 
[0102] 

5 Figure 18 is a flowchart of the face extracting 

method according to the seventh embodiment of the 
invention . 
[0103] 

In this embodiment, it is determined whether or 
10 not the face candidate area is the face using a matched 
filter within the face candidate area. 
[0104] 

First of all, the image within the face candidate 
area determined in the first embodiment is converted 

15 into the lightness image (H-l) . Next, the face 

template image is created. This image is preferably 
lightness image, with eyes, nose and mouth, in full 
face or profile, corresponding to the upper, lower, 
left or right direction of image. Also, it is 

20 preferred that several templates having different sizes 
are created. These face template images may be created 
in advance, and prestored in the storage unit 6. In 
this embodiment, since the two-dimensional Fourier 
transform is made for this face template image, the 

25 two-dimensional Fourier transform may be made in 

advance and its result may be stored in the storage 
unit 6. 
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[0105] 

And the two-dimensional Fourier transform for the 
lightness image at step (H-l) is made (H-2) . The real 
part of the result at step (H-2) and the real part of 
5 the result of the two-dimensional Fourier transform for 
the face template image are added (H-3) . Further, the 
imaginary part of the result at step (H-2) and the 
imaginary part of the result of the two-dimensional 
Fourier transform for the face template image are added 
10 (H-4). 
[0106] 

Next, the inverse Fourier transform for the real 
part and the imaginary part resulted from the addition 
is made (this process is called the matched filtering) , 
15 and this value is made the feature amount (H-8) . 
[0107] 

As the matching degree is higher, the value 
obtained by the inverse Fourier transform is larger. 
If the feature amount is more than or equal to the 
20 threshold value (H-6) , it is determined that the face 
candidate area is the face. 
[0108] 

While in the seventh embodiment as shown in 
Figure 18, the image within the face candidate area is 
25 converted into lightness image, this image may be 
converted into hue image or saturation image. 
[0109] 

- 42 - 



English translation of Japanese Patent Application Laid-Open 
No. 08-063597 

With the seventh embodiment as shown in Figure 18, 
there is the effect that each region in the face has 
high tolerance for positional differences due to 
variations among individuals, because the frequency 
5 characteristic is used. 
[0110] 

Figure 19 is a flowchart of a face extracting 
method according to an eighth embodiment of the 
invention . 
10 [0111] 

In this embodiment, the feature amount of each 
region in the face within the face candidate area is 
inputted into a neural network, and it is determined 
whether or not the face candidate area is the face. 
15 [0112] 

First of all, the feature amounts of each region 
in the face as described in other embodiments are 
obtained (J-l). The feature amounts include the 
matching degree of eyes, ratio of mouth pixel, ratio of 
20 hair pixel, and ratio of neck pixel. And these values 
are inputted into a neural network (J-2). 
[0113] 

The neural network for use has a three layer 
structure consisting of an input layer, an intermediate 
25 layer and an output layer, in which the number of 

elements in each layer is 4, 3 and 1, for example. The 
teacher data in the neural network is created in 
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advance for each pattern to cover combinations of full 
face and face in profile with the upper, lower, right 
and left direction of image. The output of teacher 
data is 1 for the face, or otherwise 0. In the neural 
5 network, the 2x4=8 patterns are learned using the 

teacher data by a back propagation method, whereby each 
coefficient is decided. Also, the configuration of the 
neural network for use is decided so that the 
recognition ratio may be highest, in which it does not 
10 matter if the number of intermediate layers is 2 or 
more and the number of elements in the intermediate 
layer is not 3. 
[0114] 

If the output from the output layer obtained in 
15 this manner is more than or equal to the threshold 

value (J-3) , it is determined that the face candidate 

area is the face (J-4). 

[0115] 

As the input into the neural network, the 
20 parameter representing the color, for example, 
lightness, hue or saturation, may be employed. 
[0116] 

In shipping an apparatus incorporating this 
extracting method, the fine adjustment is made by 
25 adding some new teacher data, if the fine adjustment is 
needed due to different environment. 
[0117] 
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With the eight embodiment as shown in Figure 19, 
the neural network is learned by some teacher data, so 
that the optimal weight coefficient can be selected for 
each input value, whereby there is the effect that the 
5 high recognition ratio can be secured. 
[0118] 

Also, even if the recognition ratio is lower due 
to different environment for the measuring system, 
there is the effect that the fine adjustment can be 
10 simply made by adding some teacher data. 
[0119] 

The face extracting method according to the 
invention can be employed not only to obtain the 
exposure value in copying the color original image onto 
15 the copying material, but also for various other kinds 
of image processing. Also, this invention can deal 
with the extraction object other than the face. 
[0120] 

[Advantages of the Invention] 
20 As described above, with the invention, the 

person's face can be extracted totally automatically 
from the color original image such as negative film or 
positive film without assistance and precisely. 



25 [Brief Description of the Drawings] 
[Figure 1] 

Figure 1 is a block diagram of a face area 
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extracting device using a face extracting method 
according to the present invention. 
[Figure 2] 

Figure 2 is a flowchart of the face extracting 
5 method according to a first embodiment of the invention. 
[Figure 3] 

Figure 3 is a view showing one example of a face 
template . 
[Figure 4] 

10 Figure 4 is a flowchart of the face extracting 

method according to a second embodiment of the 
invention . 
[Figure 5] 

Figure 5 is an explanatory view for setting an 
15 eye candidate area within the face candidate area, 

wherein Figure 5A is an example of longitudinal image 
and Figure 5B is an example of transverse image. 
[Figure 6] 

Figure 6 is an explanatory view for setting the 
20 eye candidate area within the face candidate area. 
[Figure 7] 

Figure 7 is a histogram for the lightness. 
[Figure 8] 

Figure 8 is a flowchart of the face extracting 
25 method according to a third embodiment of the invention. 
[Figure 9] 

Figure 9 is a flowchart of the face extracting 
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method according to a fourth embodiment of the 
invention . 
[Figure 10] 

Figure 10 is a flowchart of the face extracting 
5 method according to a fifth embodiment of the invention. 
[Figure 11] 

Figure 11 is explanatory views for dividing the 
face candidate area into small areas, corresponding to 
three dividing methods (a) , (b) and (c) . 
10 [Figure 12] 

Figure 12 is an explanatory view for the method 
using the color difference amount and direction of the 
average color in small area from the average color over 
the entire face candidate area as the feature amount. 
15 [Figure 13] 

Figure 13 is an explanatory view for the method 
using spatial frequency property as the feature amount. 
[Figure 14] 

Figure 14 is a flowchart of the face extracting 
20 method according to a sixth embodiment of the invention. 
[Figure 15] 

Figure 15 is a view showing an example of eye 
template, wherein Figure 15A is a view for explaining 
an eye template and its scanning direction, and Figure 
25 15B is a view showing an example of eye template with 
eyeglasses . 
[Figure 16] 

- 47 - 



English translation of Japanese Patent Application Laid-Open 
No. 08-063597 

Figure 16 is a flowchart of the face extracting 
method according to a seventh embodiment of the 
invention . 
[Figure 17] 

5 Figure 17 is an explanatory view for estimating a 

hair candidate area and a neck candidate area. 
[Figure 18] 

Figure 18 is a flowchart of the face extracting 
method according to the seventh embodiment of the 
10 invention. 
[Figure 19] 

Figure 19 is a flowchart of the face extracting 
method according to an eighth embodiment of the 
invention . 
15 [Description of Symbols] 

1 film 

2 scanner 

3 amplifier 

4 A/D converter 
20 5 CPU 

6 storage unit 
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Figure 1 

1 Film 

2 Scanner 

3 Amplifier 

5 4 A/D converter 

5 CPU 

6 Storage unit 

Figure 2 
#1 Start 

A-l Calculate feature amount of color. 
A-2 Extract flesh color area. 
A-3 Extract edge. 

A-4 Acquire template with highest matching degree for 

each candidate area. 
A-5 Is highest matching degree more than or equal to 

threshold? 
A-6 Determine face candidate area. 
#2 End 

Figure 3 

#1 Object pixel 
#2 Contour part 

25 Figure 4 

#1 Start 

B-l Set eye candidate area. 
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B-2 Create histogram of lightness within eye 

candidate area. 
B-3 Make (frequency of lightness corresponding to 
eye )/( frequency of lightness corresponding to 
5 face) feature amount. 

B-4 Is feature amount within predetermined range? 
B-5 Determine face. 
#2 End 

10 Figure 5 

(a) Example of longitudinal image 

(b) Example of transverse image 

Figure 6 
15 #1 Eye candidate area 

Figure 7 
#1 Frequency 
#2 Lightness 
20 #3 Eye portion (second peak) 

#4 Flesh color portion (first peak) 

Figure 8 
#1 Start 

25 C-l Convert face candidate area into lightness. 
C-2 Normalize size of face candidate area. 
C-3 Make Fourier transform for face candidate area. 
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C-4 Make matching degree between Fourier transform 
result of face reference data and Fourier 
transform result of face candidate area the 
feature amount. 
5 C-5 Is feature amount more than or equal to 
predetermined value? 

C-6 Determine face. 

#2 End 

10 Figure 9 

#1 Start 

D-l Compute hue and lightness of each pixel within 

face candidate area. 
D-2 Make area of covariance ellipse on hue and 
15 lightness the feature amount. 

D-3 Is feature amount within predetermined range? 
D-4 Determine face. 
#2 End 

20 Figure 10 

#1 Start 

E-l Divide face candidate area into small areas. 
E-2 Make area of covariance ellipse on hue and 

lightness in each small area the feature amount. 
25 E-3 Feature amount of upper and lower area > feature 

amount of left and right area? 
E-4 Determine face. 
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#2 End 



Figure 12 

#1 Color of lateral area 
5 #2 Entire color 

#3 Color of lower area 

#4 Lightness 

#5 Saturation 

#6 Color of upper area 

0 

Figure 13 

#1 Spatial frequency 
#2 High frequency area 



15 Figure 14 

#1 Start 

F-l Create eye template from size of face candidate 
area . 

F-2 Binarize face candidate area depending on eye or 
20 not. 

F-3 Is feature amount more than or equal to 

threshold? 
F-4 Determine face. 
#2 End 

25 

Figure 15 

#1 Eye template 
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#2 Eye + eyeglass template 

Figure 16 
#1 Start 

5 G-l Create eye template from size of face candidate 
area . 

G-2 Binarize face candidate area depending on eye or 
not . 

G-3 Make matching degree with eye template the first 
10 feature amount. 

G-4 Is first feature amount more than or equal to 

threshold? 
G-5 Determine mouth area. 

G-6 Make ratio of pixels representing mouth color the 
15 second feature amount. 

G-7 Is second feature amount within predetermined 

range? 
G-8 Determine face. 
#2 End 

20 

Figure 17 

#1 Hair candidate area 
#2 Neck candidate area 

25 Figure 18 

#1 Start 
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H-l Convert image within face candidate area into 

lightness image. 
H-2 Make Fourier transform for lightness image. 
H-3 Add real part of Fourier transform result and 

real part of reference data. 
H-4 Add imaginary part of Fourier transform result 

and imaginary part of reference data. 
H-5 Make inverse Fourier transform for added result, 

and make its value the feature amount. 
H-6 Is feature amount more than or equal to 

threshold? 
H-7 Determine face. 
#2 End 

Figure 19 
#1 Start 

J-l Compute eye matching degree, ratio of mouth 

pixels, ratio of hair pixels and ratio of neck 
pixels . 

J-2 Input into neural network. 

J-3 Is output more than or equal to threshold? 
J- 4 Determine face. 
#2 End 
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